Statnote 8: statistical power and sample size

Statistical software is now commonly available to calculate Power (P') and sample size (N) for most experimental designs. In many circumstances, however, sample size is constrained by lack of time, cost, and in research involving human subjects, the problems of recruiting suitable individuals. In addition, the calculation of N is often based on erroneous assumptions about variability and therefore such estimates are often inaccurate. At best, we would suggest that such calculations provide only a very rough guide of how to proceed in an experiment. Nevertheless, calculation of P' is very useful especially in experiments that have failed to detect a difference which the experimenter thought was present. We would recommend that P' should always be calculated in these circumstances to determine whether the experiment was actually too small to test null hypotheses adequately.

Sample size estimation and statistical power analyses

The concept of sample size and statistical power estimation is now something that Optometrists that want to perform research, whether it be in practice or in an academic institution, cannot simply hide away from. Ethics committees, journal editors and grant awarding bodies are now increasingly requesting that all research be backed up with sample size and statistical power estimation in order to justify any study and its findings. This article presents a step-by-step guide of the process for determining sample sizeand statistical power. It builds on statistical concepts presented in earlier articles in Optometry Today by Richard Armstrong and Frank Eperjesi.

The exact sample complexity of PAC-learning problems with unit VC dimension

The Vapnik-Chervonenkis (VC) dimension is a combinatorial measure of a certain class of machine learning problems, which may be used to obtain upper and lower bounds on the number of training examples needed to learn to prescribed levels of accuracy. Most of the known bounds apply to the Probably Approximately Correct (PAC) framework, which is the framework within which we work in this paper. For a learning problem with some known VC dimension, much is known about the order of growth of the sample-size requirement of the problem, as a function of the PAC parameters. The exact value of sample-size requirement is however less well-known, and depends heavily on the particular learning algorithm being used. This is a major obstacle to the practical application of the VC dimension. Hence it is important to know exactly how the sample-size requirement depends on VC dimension, and with that in mind, we describe a general algorithm for learning problems having VC dimension 1. Its sample-size requirement is minimal (as a function of the PAC parameters), and turns out to be the same for all non-trivial learning problems having VC dimension 1. While the method used cannot be naively generalised to higher VC dimension, it suggests that optimal algorithm-dependent bounds may improve substantially on current upper bounds.

Gaussian regression and optimal finite dimensional linear models

The problem of regression under Gaussian assumptions is treated generally. The relationship between Bayesian prediction, regularization and smoothing is elucidated. The ideal regression is the posterior mean and its computation scales as O(n³), where n is the sample size. We show that the optimal m-dimensional linear model under a given prior is spanned by the first m eigenfunctions of a covariance operator, which is a trace-class operator. This is an infinite dimensional analogue of principal component analysis. The importance of Hilbert space methods to practical statistics is also discussed.

Childhood ethnic differences in ametropia and ocular biometry:the Aston Eye Study

Purpose: To describe the methodology, sampling strategy and preliminary results for the Aston Eye Study (AES), a cross-sectional study to determine the prevalence of refractive error and its associated ocular biometry in a large multi-racial sample of school children from the metropolitan area of Birmingham, England. Methods: A target sample of 1700 children aged 6–7 years and 1200 aged 12–13 years is being selected from Birmingham schools selected randomly with stratification by area deprivation index (a measure of socio-economic status). Schools with pupils predominantly (>70%) from a single race are excluded. Sample size calculations account for the likely participation rate and the clustering of individuals within schools. Procedures involve standardised protocols to allow for comparison with international population-based data. Visual acuity, non-contact ocular biometry (axial length, corneal radius of curvature and anterior chamber depth) and cycloplegic autorefraction are measured in both eyes. Distance and near oculomotor balance, height and weight are also assessed. Questionnaires for parents and older children will allow the influence of environmental factors on refractive error to be examined. Results: Recruitment and data collection are ongoing (currently N = 655). Preliminary cross-sectional data on 213 South Asian, 44 black African Caribbean and 70 white European children aged 6–7 years and 114 South Asian, 40 black African Caribbean and 115 white European children aged 12–13 years found myopia prevalence of 9.4% and 29.4% for the two age groups respectively. A more negative mean spherical equivalent refraction (SER) was observed in older children (-0.21 D vs +0.87 D). Ethnic differences in myopia prevalence are emerging with South Asian children having higher levels than white European children 36.8% vs 18.6% (for the older children). Axial length, corneal radius of curvature and anterior chamber depth were normally distributed, while SER was leptokurtic (p < 0.001) with a slight negative skew. Conclusions: The AES will allow ethnic differences in the ocular characteristics of children from a large metropolitan area of the UK to be examined. The findings to date indicate the emergence of higher levels of myopia by early adolescence in second and third generation British South Asians, compared to white European children. The continuation of the AES will allow the early determinants of these ethnic differences to be studied.

Sampling and measurement methods for a study of childhood refractive error in a UK population

Background There is a paucity of data describing the prevalence of childhood refractive error in the United Kingdom. The Northern Ireland Childhood Errors of Refraction study, along with its sister study the Aston Eye Study, are the first population-based surveys of children using both random cluster sampling and cycloplegic autorefraction to quantify levels of refractive error in the United Kingdom. Methods Children aged 6–7 years and 12–13 years were recruited from a stratified random sample of primary and post-primary schools, representative of the population of Northern Ireland as a whole. Measurements included assessment of visual acuity, oculomotor balance, ocular biometry and cycloplegic binocular open-field autorefraction. Questionnaires were used to identify putative risk factors for refractive error. Results 399 (57%) of 6–7 years and 669 (60%) of 12–13 years participated. School participation rates did not vary statistically significantly with the size of the school, whether the school is urban or rural, or whether it is in a deprived/non-deprived area. The gender balance, ethnicity and type of schooling of participants are reflective of the Northern Ireland population. Conclusions The study design, sample size and methodology will ensure accurate measures of the prevalence of refractive errors in the target population and will facilitate comparisons with other population-based refractive data.

When in Rome ... ?:Human resource management and the performance of foreign firms operating in India

Purpose: The purpose of the paper is to examine the kind of HRM practices being implemented by overseas firms in their Indian subsidiaries and also to analyze the linkage between HRM practices and organizational performance. Design/methodology/approach: The paper utilizes a mixture of both quantitative and qualitative techniques via personal interviews in 76 subsidiaries. Findings: The results show that while the introduction of HRM practices from the foreign parent organization is negatively associated with performance, local adaption of HRM practices is positively related with the performance of foreign firms operating in India. Research limitations/implications: The main limitations include data being collected by only one respondent from each firm, and the relatively small sample size. Practical implications: The key message for practitioners is that HRM systems do improve organizational performance in the Indian subsidiaries of foreign firms, and an emphasis on the localization of HRM practices can further contribute in this regard. Originality/value: This is perhaps the very first investigation of its kind in the Indian context. © Emerald Group Publishing Limited.

Probing three-way interactions in moderated multiple regression:Development and application of a slope difference test

Researchers often use 3-way interactions in moderated multiple regression analysis to test the joint effect of 3 independent variables on a dependent variable. However, further probing of significant interaction terms varies considerably and is sometimes error prone. The authors developed a significance test for slope differences in 3-way interactions and illustrate its importance for testing psychological hypotheses. Monte Carlo simulations revealed that sample size, magnitude of the slope difference, and data reliability affected test power. Application of the test to published data yielded detection of some slope differences that were undetected by alternative probing techniques and led to changes of results and conclusions. The authors conclude by discussing the test's applicability for psychological research. Copyright 2006 by the American Psychological Association.

A survey of continuous improvement practices in the United Kingdom and Mexico

With the advent of globalisation companies all around the world must improve their performance in order to survive. The threats are coming from everywhere, and in different ways, such as low cost products, high quality products, new technologies, and new products. Different companies in different countries are using various techniques and using quality criteria items to strive for excellence. Continuous improvement techniques are used to enable companies to improve their operations. Therefore, companies are using techniques such as TQM, Kaizen, Six-Sigma, Lean Manufacturing, and quality award criteria items such as Customer Focus, Human Resources, Information & Analysis, and Process Management. The purpose of this paper is to compare the use of these techniques and criteria items in two countries, Mexico and the United Kingdom, which differ in culture and industrial structure. In terms of the use of continuous improvement tools and techniques, Mexico formally started to deal with continuous improvement by creating its National Quality Award soon after the Americans began the Malcolm Baldrige National Quality Award. The United Kingdom formally started by using the European Quality Award (EQA), modified and renamed as the EFQM Excellence Model. The methodology used in this study was to undertake a literature review of the subject matter and to study some general applications around the world. A questionnaire survey was then designed and a survey undertaken based on the same scale, about the same sample size, and the about the same industrial sector within the two countries. The survey presents a brief definition of each of the constructs to facilitate understanding of the questions. The analysis of the data was then conducted with the assistance of a statistical software package. The survey results indicate both similarities and differences in the strengths and weaknesses of the companies in the two countries. One outcome of the analysis is that it enables the companies to use the results to benchmark themselves and thus act to reinforce their strengths and to reduce their weaknesses.

The optimum prepaid monetary incentives for mail surveys

Increasing mail-survey response using monetary incentives is a proven, but not always cost-effective method in every population. This paper tackles the questions of whether it is worth using monetary incentives and the size of the inducement by testing a logit model of the impact of prepaid monetary incentives on response rates in consumer and organizational mail surveys. The results support their use and show that the inducement value makes a significant impact on the effect size. Importantly, no significant differences were found between consumer and organizational populations. A cost-benefit model is developed to estimate the optimum incentive when attempting to minimize overall survey costs for a given sample size. © 2006 Operational Research Society Ltd. All rights reserved.

The Effect of nutritional supplementation on subjective and objective measures of visual and retinal function:a random controlled trial

In industrialised countries age-related macular disease (ARMD) is the leading cause of visual loss in older people. Because oxidative stress is purported to be associated with an increased risk of disease development the role of antioxidant supplementation is of interest. Lutein is a carotenoid antioxidant that accumulates within the retina and is thought to filter blue light. Increased levels of lutein have been associated with reduced risk of developing ARMD and improvements in visual and retinal function in eyes with ARMD. The aim of this randomised controlled trial (RCT) was to investigate the effect of a lutein-based nutritional supplement on subjective and objective measures of visual function in healthy eyes and in eyes with age-related maculopathy (ARM) – an early form of ARMD. Supplement withdrawal effects were also investigated. A sample size of 66 healthy older (HO), healthy younger (HY), and ARM eyes were randomly allocated to receive a lutein-based supplement or no treatment for 40 weeks. The supplemented group then stopped supplementation to look at the effects of withdrawal over a further 20 weeks. The primary outcome measure was multifocal electroretinogram (mfERG) N1P1 amplitude. Secondary outcome measures were mfERG N1, P1 and N2 latency, contrast sensitivity (CS), Visual acuity (VA) and macular pigment optical density (MPOD). Sample sizes were sufficient for the RCT to have an 80% power to detect a significant clinical effect at the 5% significance level for all outcome measures when the healthy eye groups were combined, and CS, VA and mfERG in the ARM group. This RCT demonstrates significant improvements in MPOD in HY and HO supplemented eyes. When HY and HO supplemented groups were combined, MPOD improvements were maintained, and mfERG ring 2 P1 latency became shorter. On withdrawal of the supplement mfERG ring 1 N1P1 amplitude reduced in HO eyes. When HO and HY groups were combined, mfERG ring 1 and ring 2 N1P1 amplitudes were reduced. In ARM eyes, ring 3 N2 latency and ring 4 P1 latency became longer. These statistically significant changes may not be clinically significant. The finding that a lutein-based supplement increases MPOD in healthy eyes, but does not increase mfERG amplitudes contrasts with the CARMIS study and contributes to the debate on the use of nutritional supplementation in ARM.

Statnote 13: a more complex analysis of variance incorporating a 'repeated measures' factor

Experiments combining different groups or factors and which use ANOVA are a powerful method of investigation in applied microbiology. ANOVA enables not only the effect of individual factors to be estimated but also their interactions; information which cannot be obtained readily when factors are investigated separately. In addition, combining different treatments or factors in a single experiment is more efficient and often reduces the sample size required to estimate treatment effects adequately. Because of the treatment combinations used in a factorial experiment, the degrees of freedom (DF) of the error term in the ANOVA is a more important indicator of the ‘power’ of the experiment than the number of replicates. A good method is to ensure, where possible, that sufficient replication is present to achieve 15 DF for the error term of the ANOVA testing effects of particular interest. Finally, it is important to always consider the design of the experiment because this determines the appropriate ANOVA to use. Hence, it is necessary to be able to identify the different forms of ANOVA appropriate to different experimental designs and to recognise when a design is a split-plot or incorporates a repeated measure. If there is any doubt about which ANOVA to use in a specific circumstance, the researcher should seek advice from a statistician with experience of research in applied microbiology.

Data analysis methods in optometry Part 7: multiple linear regression

Multiple regression analysis is a complex statistical method with many potential uses. It has also become one of the most abused of all statistical procedures since anyone with a data base and suitable software can carry it out. An investigator should always have a clear hypothesis in mind before carrying out such a procedure and knowledge of the limitations of each aspect of the analysis. In addition, multiple regression is probably best used in an exploratory context, identifying variables that might profitably be examined by more detailed studies. Where there are many variables potentially influencing Y, they are likely to be intercorrelated and to account for relatively small amounts of the variance. Any analysis in which R squared is less than 50% should be suspect as probably not indicating the presence of significant variables. A further problem relates to sample size. It is often stated that the number of subjects or patients must be at least 5-10 times the number of variables included in the study.5 This advice should be taken only as a rough guide but it does indicate that the variables included should be selected with great care as inclusion of an obviously unimportant variable may have a significant impact on the sample size required.

The degradation of gel-spun poly(beta-hydroxybutyrate) fibrous matrix

Poly(β-hydroxybutyrate), (PHB), is a biologically produced, biodegradable thennoplastic with commercial potential. In this work the qualitative and quantitative investigations of the structure and degradation of a previously unstudied, novel, fibrous form of PHB, were completed. This gel-spun PHB fibrous matrix, PHB(FM), which has a similar appearance to cotton wool, possesses a relatively complex structure which combines a large volume with a low mass and has potential for use as a wound scaffolding device. As a result of the intrinsic problems presented by this novel structure, a new experimental procedure was developed to analyze the degradation of the PHB to its monomer hydroxybutyric acid, (HBA). This procedure was used in an accelerated degradation model which accurately monitored the degradation of the undegraded and degraded fractions of a fibrous matrix and the degradation of its PHB component. The in vitro degradation mechanism was also monitored using phase contrast and scanning electron microscopy, differential scanning calorimetry, fibre diameter distributions and Fourier infra-red photoacoustic spectroscopy. The accelerated degradation model was used to predict the degradation of the samples in the physiological model and this provided a clearer picture as to the samples potential biodegradation as medical implantation devices. The degradation of the matrices was characterized by an initial penetration of the degradative medium and weakening of the fibre integrity due to cleavage of the ester linkages, this then led to the physical collapse of the fibres which increased the surface area to volume ratio of the sample and facilitated its degradation. Degradation in the later stages was reduced due to the experimental kinetics, compaction and degradation resistant material, most probably the highly crystalline regions of the PHB. The in vitro degradation of the PHB(FM) was influenced by blending with various polysaccharides, copolymerizing with poly(~-hydroxyvalerate), (PHV), and changes to the manufacturing process. The degradation was also detennined to be faster than that of conventional melt processed PHB based samples. It was concluded that the material factors such as processing, sample size and shape affected the degradation of PHB based samples with the major factor of sample surface area to volume ratio being of paramount importance in determining the degradation of a sample.

Development and validaton of an indicator to compare antibiotic usage in secondary care in England

The use of antibiotics was investigated in twelve acute hospitals in England. Data was collected electronically and by questionnaire for the financial years 2001/2, 2002/3 and 2003/4. Hospitals were selected on the basis of their Medicines Management Self-Assessment Scores (MMAS) and included a cohort of three hospitals with integrated electronic prescribing systems. The total sample size was 6.65% of English NHS activity for 2001/2 based on Finished Consultant Episode (FCE) numbers. Data collected included all antibiotics dispensed (ATC category J01), hospital activity FCE's and beddays, Medicines Management Self-assessment scores, Antibiotic Medicines Management scores (AMS), Primary Care Trust (PCT) of origin of referral populations, PCT antibiotic prescribing rates, Index of Multiple Deprivation for each PCT. The DDD/FCE (Defined Daily Dose/FCE) was found to correlate with the DDD 100beddays (r = 0.74 psample mean at 3.48 DDD/FCE in 2001/2 and 3.34 DDD/FCE in 2003/4. The MMAS and AMS were found to correlate (r = 0.74 p